21 November, 2023
This document explores the results of expert opinion gathered from staff at the Woodland Trust, related to the measurement of woodland ecological condition by a set of 14 indicators.
Here, we estimate the value function and weight for each indicator using a method of expert opinion elicitation called the Delphi method. The Delphi method is designed to collect and distill expert knowledge, using iterative or repetitive surveys to increase robustness . The method is also useful for identifying where agreements and disagreements occur.
The method relies on repetitive surveying in at least two to three‘rounds’ where participants are asked the same questions, to which they provide quantitative answers. After each round participants review and consider the answers given by all panel-members, and are then given the opportunity to revise their answers. Results are considered relatively robust when they change little between rounds. The Delphi method has a history of successful use in supporting conservation action where empirical evidence is insufficient and information is needed rapidly to inform decisions. It’s transparency and repeatability also add to its apeal.
Each participant either attended an introductory workshop to the process, or was taken through individually. A sub-section of participants were also involved in development the proposed methods to measure each indicator in the field. Following this participants were supplied with an Excel spread sheet in which to submit their answers to questions relating to:
Participants were encouraged to keep a clear distinction between:
It was also stated that noted exceptions would be analysed and, where appropriate, used to inform future development of the WEC measure to account for important nuance.
A set of 14 structural indicators were proposed to measure WEC.
Values and weights are all measured relative to each other, so each participant must be using the same scale. Some respondents did not use 100 and 0 as the highest and lowest scores for all indicator value functions and weights. To make their relative values comparable to other indicators all value functions provided where scaled to 0-100. Likewise, weightings were scaled so that the highest indicator weighting for each panelist was 100.
Value functions scaled for:
- Adam Th - Number deadwood classes present (of 3 potential classes)
across 4 plot-quarters,
- Adam Th - Number of native tree and shrub
species,
- Adam Th - Number of tree size classes (age proxy),
-
Adam Th - Vertical structure: Number of tree and shrub canopy layers
present,
- Alasdair Fi - Number deadwood classes present (of 3
potential classes) across 4 plot-quarters,
- Alasdair Fi - Number of
tree size classes (age proxy),
- Alasdair Fi - Tree regeneration,
- Alasdair Fi - Vertical structure: Number of tree and shrub canopy
layers present,
- Chris R - Number of ancient/veteran trees per 1 ha
plot,
- Chris R - Number of tree size classes (age proxy),
-
Dave Bo - Extent/ area of woodland,
- Dave Bo - Number deadwood
classes present (of 3 potential classes) across 4 plot-quarters,
-
Dave Bo - Number of ‘positive indicator’ plants per plot (10m radius
circle),
- Dave Bo - Number of ancient/veteran trees per 1 ha plot,
- Dave Bo - Number of native tree and shrub species,
- Dave Bo -
Number of tree size classes (age proxy),
- Dave Bo - Tree
regeneration,
- Dave Bo - Vertical structure: Number of tree and
shrub canopy layers present,
- Emma G - Number of native tree and
shrub species,
- Emma G - Number of tree size classes (age proxy),
- Emma G - Occupancy of native trees & shrubs in all canopy
layers,
- Jim Sm-Wr - Extent/ area of woodland,
- Jim Sm-Wr -
Horizontal complexity (structural mosaics across a wood),
- Jim
Sm-Wr - Invasive plant species presence and cover,
- Jim Sm-Wr -
Number of ‘positive indicator’ plants per plot (10m radius circle),
- Jim Sm-Wr - Number of native tree and shrub species,
- Jim
Sm-Wr - Number of tree size classes (age proxy),
- Jim Sm-Wr -
Occupancy of native trees & shrubs in all canopy layers,
- Jim
Sm-Wr - Vertical structure: Number of tree and shrub canopy layers
present,
- Kylie Jo-Ma - Extent/ area of woodland,
- Kylie Jo-Ma
- Herbivore impact,
- Kylie Jo-Ma - Number of ‘positive indicator’
plants per plot (10m radius circle),
- Kylie Jo-Ma - Number of
ancient/veteran trees per 1 ha plot,
- Kylie Jo-Ma - Number of
native tree and shrub species,
- Kylie Jo-Ma - Number of tree size
classes (age proxy),
- Kylie Jo-Ma - Occupancy of native trees &
shrubs in all canopy layers,
- Kylie Jo-Ma - Tree regeneration,
- Kylie Jo-Ma - Vertical structure: Number of tree and shrub canopy
layers present,
- Liam Pl - Extent/ area of woodland,
- Liam Pl
- Herbivore impact,
- Liam Pl - Number of native tree and shrub
species,
- Liam Pl - Number of tree size classes (age proxy),
-
Liam Pl - Occupancy of native trees & shrubs in all canopy layers,
- Liam Pl - Tree regeneration,
- Lou Ha - Number of tree size
classes (age proxy),
- Martin Hu - Horizontal complexity (structural
mosaics across a wood),
- Martin Hu - Number deadwood classes
present (of 3 potential classes) across 4 plot-quarters,
- Martin Hu
- Occupancy of native trees & shrubs in all canopy layers,
-
Mick Br - Anthropogenic damage,
- Mick Br - Herbivore impact,
-
Mick Br - Horizontal complexity (structural mosaics across a wood),
- Mick Br - Invasive plant species presence and cover,
- Mick Br
- Number deadwood classes present (of 3 potential classes) across 4
plot-quarters,
- Mick Br - Number of ‘positive indicator’ plants per
plot (10m radius circle),
- Mick Br - Number of ancient/veteran
trees per 1 ha plot,
- Mick Br - Number of native tree and shrub
species,
- Mick Br - Number of tree size classes (age proxy),
-
Mick Br - Occupancy of native trees & shrubs in all canopy layers,
- Mick Br - Tree disease and rapid mortality,
- Mick Br - Tree
regeneration,
- Peter Lo - Extent/ area of woodland,
- Peter Lo
- Herbivore impact,
- Peter Lo - Horizontal complexity (structural
mosaics across a wood),
- Peter Lo - Number of ‘positive indicator’
plants per plot (10m radius circle),
- Peter Lo - Number of
ancient/veteran trees per 1 ha plot,
- Peter Lo - Number of native
tree and shrub species,
- Peter Lo - Number of tree size classes
(age proxy),
- Peter Lo - Vertical structure: Number of tree and
shrub canopy layers present,
- Rich Br - Number of tree size classes
(age proxy),
- Saul H - Extent/ area of woodland,
- Saul H -
Horizontal complexity (structural mosaics across a wood),
- Saul H -
Number of tree size classes (age proxy),
- Vanessa B - Herbivore
impact,
- Vanessa B - Number of tree size classes (age proxy)
Weights were scaled for:
- Chris R,
- Dave Bo,
- Hannah Pa,
- Iain Mo,
-
Jim Sm-Wr,
- Liam Pl,
- Saul H
Between rounds, expert-opinion estimates were presented on interactive plots to allow respondents to investigate estimates provided by others, and their certainties.
Value functions where plotted with lines connecting each of the points provided, and a model (details below) was fitted to present the average trend, where a consensus was deemed to be emerging. Hover-text displayed respondent name, their certainty in their estimate, and a sentence provided that describes the trend.
Indicator relative weightings were presented on a dot-plot, including the mean and median results for each indicator’s weight.
Average value functions were modeled with a binomial GAM. The influence of points from each respondent weighted by the inverse of the number of points provided by that respondent, with the aim of ensuring that all respondents had the same influence on the result, regardless of how many points they provided.
27 expert practitioners from Woodland Trust and Plantlife staff were
invited to provide their opinion the value and weight of
indicators.
20 have provided a response as of Tue Nov 21 16:27:28 2023.
Of these, 12 remain partially incomplete.
The chart below summarises participant completion of different elements
of the survey. For more detail see table (button below chart).
Figure 1.
Completion of expert opinion survey components by each panelist
(coloured portion of pies) for value functions (“Value func”), value
function certainties (“VF cert”), indicator weightings (“Weights”) and
weighting certainties (“wt certs”).
Warning - table that should be printed here is written, just not printing
[1] “Number of tree size classes (age proxy)” [1] “17 respondants.”
| measure | Mean | Standard deviation |
|---|---|---|
| 1 | 0.0 | 0.0 |
| 2 | 40.0 | 15.5 |
| 3 | 76.5 | 15.0 |
| 4 | 100.0 | 0.0 |
Comments
Consensus on VF likely:
- All continuous relationships have same shape.
- Complete agreement on upper and lower value.
- Some put maximum value at 3 classes.
Disagreement on weight:
- Two camps, putting average weights at c. 60 and 90, respectively.